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dUX Remote Sensing Data Characterizations 

EfcMj— W = Stennis St 


Passive Electro-Optical Systems 


Stennis Space Center 


• To make sound decisions based on remotely sensed data, 
scientists and others who use the data need to have a high level 
of confidence in the data 

• Characterizations fall into 3 categories 

- Geometric (absolute and relative) 

- Spatial (GSD, image sharpness-MTF, edge response, ...) 

- Radiometric (absolute and relative) 

• Characterizations become particularly important when using 
commercially obtained imagery 

- Designed and operated outside the NASA scientific 
community 

• A single site can be used to characterize high-spatial resolution 
imagery 

- Increased spatial resolution and decreased swath 


March 8, 2006 


y 


NASA 

V J ■ 


NASA Stennis Space Center, MS 

Stennis Space Center 


Site: 55 km 2 , scattered buildings 
within a heavily wooded area, 
manmade reservoirs and canals. 



Nicholson Quad 


Dead Tiger Creek Quad 



Haaswood Quad 


Logtown Quad 


Elevation: 5.5 m -10 m 
Centerpoint: 30.39° N, 89.61° W 
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Geometric Characterizations 
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Geometric Characterization 

Stennis Space Center 


• Site Requirements 

- Array of at least 20 positionally known and identifiable 
points located evenly throughout the image acquisition 
area 

• Known to an order of magnitude better than the spatial 
resolution of the system being characterized 

• National Standard for Spatial Data Accuracy (NSSDA) 

• Characterization Technique 

- Compare georeferenced image pixels to surveyed 
points to determine check point error 

- Calculate measure of merit 

• ce 95 
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SSC Geolocation Targets - Ground Control 

Stennis Space Center 


SSC Geodetic Targets 

- 45 targets currently deployed 

- 2.44-m diameter, painted white 

- 0.6-m diameter, center painted red 

- Target centers geolocated by GPS 
to within 3 cm horizontal CE 90 and 
9 cm vertical LE 90 



Painted Manhole Covers 

- 136 painted manhole covers 

- -0.65 paint reflectivity 

- 0.6- to 2.9-m diameter 

- Target centers geolocated by 
GPS to within 3 cm horizontal 
CE90 and 9 cm vertical LE 90 
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Ground^Con_trol_Po[nt_SurveY 


Global Positioning System (GPS) 
survey of ground control points 

Phase differencing receiver 
operating in real time kinematic 
(RTK) mode 
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SSC Target Layout 


Stennis Space Center 
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Check Point Error 


Stennis Space Center 


Check Point Error - differences between 
image and reference coordinates 

AX = X -X , 

i image, i reference, i 

A Y=Y -7, 

i image, i reference, i 

Check point error radial 
magnitude calculated by 


ARi = y /AX i 2 +AY i : 


Reference 

Coordinate 
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Circular Error Definitions 


• CE 90 - The radial error which 90% of all errors in a circular distribution 
will not exceed (adapted from Greenwalt and Shultz, 1962) 

- Equivalent to the Circular Map Accuracy Standard (CMAS) 

• CE 95 - The radial error which 95% of all errors in a circular distribution 
will not exceed (adapted from Greenwalt and Shultz, 1962) 

- Equivalent to Accuracy r (from NSSDA) 

• In the normal case, circular error may be generally defined as the circle 
radius, R, that satisfies the conditions of the equation below (where C.L. 
is the desired confidence level); however, there is no analytical solution 
to this equation. 



• Empirical approach to CE 90 where CE 90 = 90 th percentile of AR 

Sources: Ross, K.., 2005. Geopositional statistical methods. In Proceedings of the 2004 High Spatial Resolution Commercial Imagery Workshop, 

NASA/NGA/USGS. U.S. Geological Survey National Center, Reston, VA, November 8-10, 2004. John C. Stennis Space Center, MS: 
National Aeronautics and Space Administration. CD-ROM. 

Greenwalt, C.R., and M.E. Shultz, 1962. Principles of Error Theory and Cartographic Applications. ACIC Technical Report No. 96, United 
March 8, 2006 States Air Force, Aeronautical Chart and Information Center, St. Louis, Missouri, 98 pp. 
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^Example Geometric Characterization - DMC 

f— ^ ^ Stermis Space Center 

Vector Plot (AR) 


• Standard mosaicked 
georeferenced image 
product 

• 3001, Inc., generated 
using data captured 
with a Z/l Imaging 
Digital Mapping 
Camera (DMC) 

• Acquired over SSC 
November 8, 2004 


Stennis Geometric Assessment Vector Plot 



The calculations for the 0.1524 m (6 in) GSD product indicate a 
CE 90 of 0.39 m, a CE 95 of 0.48 m, and a net RMSE of 0.27 m 
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Example Geometric Characterization 


- OV-3 

Stennis Space Center 


Vector Plot (AR) 


OrbView-3 

Acquisition 9/17/2003 

Date 

Imagery Band PAN 

i i 

Number Tare 

jets Used 

Error Com| 

aonents 

n 

| 40 

Ph (Bias) 

7.92 m 

Test for Departure from 
Circular Distribution 

o c (Circular 
Standard 

Error) 

0.64 m 

St. Dev. Min 
Max Ratio 

0.84 


12.40 

St. Dev. Min Max Ratio should 
be at least 0.6 for Circular 
Error assumptions. 

If p H /tfc is greater than 0.1, 
then error calculations should 
account for bias. 


Circular Error 

Empirical CE 90 

8.32 m 

Empirical CE 95 

8.44 m 


CE 90 8.32 m 


6 Geopositional Assessment Vector Plot 



March 8, 2006 


13 




Stennis Space Center 


Example Scatterplot_^_OV-3 


• Position of 
point clusters 
away from (0,0) 
indicate 
absolute error 


• Position of 
point within a 
cluster indicate 
relative error 
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Spatial Characterizations 
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Spatia l Characterization 


• Site Requirements - Edge Analysis 
- High-contrast edge 

• Contrast should strive to maximize the dynamic range of the sensor 
being evaluated 

• Sized such that at least 30 pixels lie across the edge (10 bright, 10 dark, 
10 transition edge) 

• Edges should be formed in two perpendicular planes 


• Characterization Technique 

- Edges tilted 4-8 degrees from image frame of reference 

- Edge response plotted 

- Calculate measure of merit - “sharpness” parameter 

• MTF (Modulation Transfer Function) at Nyquist frequency 

• FWHM (Full Width Half Maximum) of line spread function 

• RER (Relative Edge Response) 
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Tilted Edge Technique 




Stennis Space Center 



8 - pixel index 


X - pixel’s distance from edge (in GSD) 


Problem: Digital cameras 
undersample edge target 

Solution: Image tilted edge to 

improve sampling 
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SSC Painted Concrete Edge Targets 




Stennis Space Center 
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Modu lat i on Transfer Function (MTF) 

^^ # Sfenn/'s Space Center 



-15 -10 -5 o 5 10 15 



-5 -4 -3 -2 -1 0 1 2 3 4 5 

Distance / GSD 


Measured edge response 
along “tilted edge” 


Derivative of edge 
response or line spread 
function 



• Fourier transform of line 
spread function or MTF 

• Nyquist frequency is 0.5 * 
sampling frequency or 
(1/(2GSD)) 
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Relative Edge Response 


Stennis Space Center 



• Another measure of spatial 
resolution is a difference of 
normalized edge response 
values at points distanced from 
the edge by -0.5 and 0.5 GSD 

• Relative Edge Response is one 
of the engineering parameters 
used in the General Image 
Quality Equation to provide 
predictions of imaging system 
performance expressed in terms 
of the National Imagery 
Interpretability Rating Scale. 


RER = ^\ER X ( 0 . 5 ) - ER X (- 0 . 5 )][£ 7 ?>. ( 0 . 5 ) - ER y (- 0 . 5 )] 
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Meaning of RER in Remote Sensing 


Stennis Space Center 


Radiance measured for each pixel is 
assumed to come from the Earth’s 
surface area represented by that pixel. 
However, because of many factors, 
actual measurements integrate 
radiance L from the entire surface with 
a weighting function provided by a 
system’s point spread function ( PSF): 

00 00 

L r =\ Jpsf ( x,y)L(x,y)dxdy 

— 00 —00 

Part of radiance that originates in the 
pixel area is given by: 

0.5 0.5 

L p = | | PSF(x,y)L(x,y)dxdy 

- 0.5 - 0.5 

Relative Edge Response squared 
(RER 2 ) can be used to assess the 
percentage of the measured pixel 
radiance that actually originates from 
the Earth’s surface area represented 
by the pixel: 


L p / L t « 


RER 


A simple example: 
Box PSF 
Width = 2 GSD 



ER(0.5) - ER(-0.5) = 
0.75 - 0.25 = 0.50 

RER = 0.50 


-1 0 1 
Distance / GSD 



RER 2 = 0.25 means that 
25% of information collected 
with the pixel PSF (blue 
square) comes from the 
actual pixel area (shadowed 
square) 


Distance / GSD 

GSD 

◄ — ► 



Source: Blonski, S., 2005. Spatial resolution characterization for QuickBird image products: 2003-2004 season. In Proceedings of the 2004 High Spatial 
Resolution Commercial Imagery Workshop, NASA/NGA/USGS. U.S. Geological Survey National Center, Reston, VA, November 8-10, 
2004. John C. Stennis Space Center, MS: National Aeronautics and Space Administration. CD-ROM. 
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Example Spatial Characterization - RER 

— ^ ^ Stennis Space Center 


QuickBird panchromatic Imagery 
acquired January 10, 2004 


04jan1 01 63033-p2as-0000000981 96_01 _p001 _TARPS.tif 



04jan10163033-p2as-000000137430_01_p001_TARPS.tif 



Cubic Convolution resampling 
RER = 0.5 


MTF Boost resampling 
RER = 0.79 
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Examp l e_Spatjal C h aracterization^Q B 


Stennis Space Center 


Image 

Tracking 

ID 

Acquisition Site 

Acquisition 

Date 

Satellite Angle (°) 

Resampled 
GSD (m) 

Resampling 

Method 

Target 

Direction 

MT^Nyquist 

RER 

Zenith 

Azimuth 

88502 

Brookings, SD 

2003-09-15 

6.7 

287.2 

0.6 

CC 

Easting 

0.14 ± 0.01 

0.51 ±0.01 

98196 

SSC, MS 

2004-01-10 

0.6 

244.0 

0.6 

cc 

Easting 

0.14 ±0.02 

0.54 ±0.01 

102569 

SSC, MS 

2004-01-28 

15.4 

8.9 

0.6 

CC 

Northing 

0.11 ±0.01 

0.49 ±0.01 

76412 

Brookings, SD 

2003-09-15 

6.7 

287.2 

0.6 

MTF 

Easting 

0.55 ±0.07 

0.84 ±0.01 

137430 

SSC, MS 

2004-01-10 

0.6 

244.0 

0.6 

MTF 

Easting 

0.50 ±0.01 

0.81 ±0.01 

102569 

SSC, MS 

2004-01-28 

15.4 

8.9 

0.6 

MTF 

Northing 

0.41 ± 0.04 

0.76 ±0.01 


Spatial characterization summary of QuickBird panchromatic imagery 
acquired at SSC and at Brookings, SD, in the 2003-2004 time frame 
(± uncertainty estimated from standard deviation of multiple results). 
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Radiometric Characterization 


Reflective Region 


Stennis Space Center 


• Site Requirements 

- Uniform Radiometric Targets 

• Manmade or naturally occurring 

• Sufficiently large to reduce/eliminate adjacency effects 

- Target reflectance measurement capability 

- Atmospheric measurement capability 

• Aerosols 

• Water vapor 

- Radiative transport understanding/code capability 

• MODTRAN 

• Characterization Technique 

- Reflectance-based vicarious calibration approach 
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Reflectance-based 

Stennis Space Center 

Vicarious Calibration Approach 


• Measure target/ground reflectance coincident with the satellite 
acquisition 


• Measure atmospheric aerosols, and pressure, temperature, and water 
vapor profiles coincident with the satellite acquisition 

• Use these measurements along with acquisition geometry/location 
parameters as input into a radiative transfer model to predict at-sensor 
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NASA SSC Target Field 


Stennis Space Center 


QuickBird image acquired 
January 10, 2004 
True-Color Pan-Sharpened 
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Radiometric Tarps 


• Four 20-m x 20-m tarps with reflectance 
values of approximately 3.5%, 22%, 34%, 
and 52% 


• Spectral measurement range of 400 to 
1050 nm 


• Standard deviation about average 
reflectance less than 1% spatially 

• Peak-to-peak variation in reflectance less 
than 10% within any 100-nm spectral 
band 

• Less than 10% variation in reflectance 
values when measuring tarps from 10° to 
60° off axis 

• Each side is straight to within ±6.0 cm 
over the 20-m length 


Manufactured by 
MTL Systems, Inc. / Group VIII 
Technology, Inc. 



• Each tarp panel has 60 square witness 
samples measuring 30.5 cm by 30.5 cm 
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Reflectance Measurements 


• ASD FieldSpec FR Spectroradiometers 
measure spectral reflectance and radiance of 
99% reflectance Spectralon® panels (modified 
Jackson BRDF model) and radiometric targets 

• ASD FieldSpec FR Spectroradiometers are 
radiometrically calibrated in the SSC Instrument 
Validation Lab (IVL) before field use 



1.0 


Spectral Albedo for Stennis Space Center, 2/17/02 


0.9 
0.8 
0.7 

0 ) 

c 0.6 
| 0.5 

*4- 
0 ) 

0£ 0.4 
0.3 
0.2 
0.1 

0 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 

Wavelength (um) 


52% tarp 
22% tarp 
3.5% tarp 
Grass 
Concrete 


All measurements taken 
within ±20 minutes of 
satellite overpass 

Most acquisition dates 
utilized multiple ASDs for 
cross comparison 

Typically 1000+ spectra 
are averaged for each 
target 
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Calibration and Characterization of 

Stennis Space Center 

ASD FieldSpec Spectroradiometers 


• NASA SSC maintains four ASD 
FieldSpec FR spectroradiometers 

- Laboratory transfer radiometers 

- Ground surface reflectance for V&V 
field collection activities 

• Radiometric Calibration 

- NIST-calibrated integrating sphere 
serves as source with known spectral 
radiance 

• Spectral Calibration 

- Laser and pen lamp illumination of 
integrating sphere 

• Environmental Testing 

- Temperature stability tests performed in 
environmental chamber 
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Laboratory BRDF Measurements 


Stennis Space Center 


• Purpose 

- Laboratory BRDF measurements 
are used to correct ground-based 
reflectance measurements for 
satellite viewing and for solar 
illumination geometry 

• Method 

- Collimated FEL lamp source 

- NIST-calibrated Spectralon panel 
serves as reference 

- Goniometer-mounted sample 
controls illumination geometry 

- Optronics OL750 hyperspectral 
instrument measures spectra 
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Atmospheric Measurements 


• Cloud cover monitoring - Total Sky Imager 

• Direct solar irradiance - Automated Solar Radiometer 

• Direct, total, and diffuse irradiance - Multi-filter Rotating Shadow-band 
Radiometer 

• Direct, total, and diffuse radiance - ASD Spectroradiometer/Spectralon 
white reference 

• Vertical profiles of temperature, pressure and relative humidity - 


Radiosonde Balloon 
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Cross Compajjson of Atmospheric 

Data 


Stennis Space Center 


Optical depth comparisons are 
made between fielded 
instruments that have different 
channels and measurement 
principles. 



Q. 

© 

Q 

ro 

o 

Q. 

o 


MFRSR/ASR Optical Depth Values for SSC 01-15-2002 


O ASR #25 SSC Field 
■ MFRSR #451 SSC Roof 
A MFRSR #477 SSC Field 
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Vi sibility and Aerosol Asymmetry 

Estimation 


Stennis Space Center 


ASR and MODTRAN Predicted Transmissions for Stennis, 4/15/05 



0.4 - 


0.3 - ; i : : : ■: - 

0 2 i i i i i i i 

'300 400 500 600 700 800 900 1000 1100 

Wavelength (nm) 


Visibility is estimated by 
comparing MODTRAN output 
transmission values to ASR 
measured values 


MODTRAN and MFRSR Diffuse to Global (D2G) Ratios for Stennis Space Center, 01/10/04 



The asymmetry factor for the 
aerosol scattering phase 
function is estimated by 
comparing MODTRAN output 
diffuse-to-global ratio values to 
MFRSR measured diffuse-to- 
global ratio values 
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Comparison to Spectrajon Panel 


Stennis Space Center 


• Verification of parameters used to generate MODTRAN at-sensor radiance estimate 

- Measuring the radiance of Spectralon panel with a well-calibrated spectroradiometer 
is a way of measuring atmospheric global and diffuse irradiance 

- Use ground truth data and geometry modeling an ASD FieldSpec FR 
spectroradiometer measuring a 99% reflectance Spectralon panel as input to 
MODTRAN to predict radiance 

- Compare MODTRAN-calculated radiance to actual radiance measured from 
Spectralon panel to verify the atmospheric model 
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Example IKONOS Data Acquisitions 

Stennis Space Center 


Site/Date 

Overpass 
rime (UTC) 

Satellite 

Elevation 

Stennis 12/15/04 

16:45 

68.9 deg 

Wiggins 3/24/05 

16:50 

86.3 deg 

Stennis 4/15/05 

16:51 

72.7 deg 

Standard imagery 
Cubic Convolution 

resampling, MTFC Off 


Satellite 

Sun 

Sun 

Azimuth 

Elevation 

Azimuth 

118.6 deg 

34.0 deg 

160.8 deg 

71.9 deg 

56.3 deg 

146.1 deg 

25.4 deg 

64.5 deg 

138.8 deg 



Stennis Space Center, MS, 12/15/04 
March 8, 2006 




Stennis Space Center, MS, 4/15/05 
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IKONOS Spectraj_Response 


IKONOS Relative Spectral Response 



350 
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950 


Wavelength (nm) 
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2004/2005 IKONOS Calibration Resul ts 

Stennis Space Center 


Blue Band Calibration Summary 


Stennis, 52%tarp,1 2/1 5/04 
Stennis, 34%tarp, 1 2/1 5/04 
Stennis, 22%tarp, 1 2/1 5/04 
Wiggins, 52%tarp, 3/24/05 
Stennis,52%tarp, 4/1 5/05 
Stennis, 34%tarp, 4/1 5/05 
Stennis, 22%tarp, 4/1 5/05 

NASA Cal Curve 

NASA Cal Curve! la 



-NASA Radiance = DN / 67.1 ± 1 .5 


DN 


Green Band Calibration Summary 


Stennis, 52%tarp, 1 2/1 5/04 
Stennis, 34%tarp, 1 2/1 5/04 
Stennis, 22%tarp, 1 2/1 5/04 
Wiggins, 52%tarp, 3/24/05 
Stennis,52%tarp, 4/1 5/05 
Stennis, 34%tarp, 4/1 5/05 
Stennis, 22%tarp, 4/1 5/05 

NASA Cal Curve 

NASA Cal Curve! la 


r-NASA Radiance = DN / 70.0 ± 1 .6 



Red Band Calibration Summary 


Stennis, 52%tarp,1 2/1 5/04 
Stennis, 34%tarp, 12/1 5/04 
Stennis, 22%tarp, 1 2/1 5/04 
Wiggins, 52%tarp, 3/24/05 
Stennis,52%tarp, 4/1 5/05 
Stennis, 34%tarp, 4/1 5/05 
Stennis, 22%tarp, 4/1 5/05 

- NASA Cal Curve 

- NASA Cal Curve! la 



ASA Radiance = DN / 91 .2 ± 1 .7 


NIR Band Calibration Summary 


Stennis, 52%tarp, 1 2/1 5/04 
Stennis, 34%tarp,1 2/1 5/04 
Stennis, 22%tarp, 1 2/1 5/04 
Wiggins, 52%tarp, 3/24/05 
Stennis, 52%tarp, 4/1 5/05 
Stennis, 34%tarp, 4/1 5/05 
Stennis, 22%tarp, 4/1 5/05 

- NASA Cal Curve 

- NASA Cal Curve! la 



^NASA Radiance = DN / 82.5 ± 1 .8 
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IKONOS Calibration Coefficients and 

Stennis Space Center 

Associated Uncertainty 


Band 

NASA 

Estimate 

2000* 

NASA 

Estimate 

2000 

Scaled 

NASA 
Estimate 
2001 and 
% Change 

NASA 
Estimate 
2002 and 
% Change 

NASA 
Estimate 
2004/2005 and 
% Change 

Blue 

64.2 ± 4.3 

73.9 ±4.9 

73.2+4.7 

-0.9% 

71.0 ±4.7 
-3.9% 

67.1 ± 1.5 
-9.2% 

Green 

65.4 ± 4.2 

73.3 ±4.7 

76.6 ± 3.8 
4.5% 

73.4 ± 5.0 
0.2% 

70.0 ± 1.6 
-4.5% 

Red 

88.1 ± 7.0 

99.5 ± 7.9 

101.8 ±5.3 
2.3% 

97.5 ± 7.7 
-2.0% 

91.2 ± 1.7 
-8.4% 

NIR 

73.7 ± 3.8 

83.3 ±4.3 

85.9 ±4.2 
3.1% 

82.7 ± 5.8 
-0.7% 

82.5 ± 1.8 
-1.0% 


Calibration coefficients in units of DN / ( W / m 2 sr ) 

Uncertainty defined as 1 -sigma 

% change relative to NASA estimate year 2000 scaled data 
* denotes NASA estimate made before Space Imaging’s image processing upgrade 
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5-Year IKONOS Calibration Summary 


tennis Space Center 


5-Year IKONOS Blue Band Calibration 


5-Year IKONOS Green Band Calibration 
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NASA Calibration Team Benefits 


High Spatial Resolution Commercial Imagery Evaluation 


• Three independent groups employed similar approaches but 
different tools and techniques. 

- Checks and balances between groups 

- Removes/reduces any bias associated with one individual 
group or set of techniques 

• Multiple study sites were used 

- Removes/reduces any bias associated with a single study site 

- Radiance values found within these study site scenes span 
the dynamic range of the sensor. 




SDSU 
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Summary 


Stennis Space Center 


• NASA Stennis Space Center has developed a Verification 
and Validation site capable of performing geometric, spatial, 
and radiometric characterizations of high spatial resolution 
imagery 

- Laboratory calibration facility 

- Field targets and infrastructure 

- Data processing algorithms and techniques 

• SSC collaborates with other recognized V&V teams for 
independent checks/balances 


March 8, 2006 
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